Articulated Clinician Detection Using 3D Pictorial Structures on RGB-D Data

Reliable human pose estimation (HPE) is essential to many clinical applications, such as surgical workflow analysis, radiation safety monitoring and human-robot cooperation. Proposed methods for the operating room (OR) rely either on foreground estimation using a multi-camera system, which is a challenge in real ORs due to color similarities and frequent illumination changes, or on wearable sensors or markers, which are invasive and therefore difficult to introduce in the room. Instead, we propose a novel approach based on Pictorial Structures (PS) and on RGB-D data, which can be easily deployed in real ORs. We extend the PS framework in two ways. First, we build robust and discriminative part detectors using both color and depth images. We also present a novel descriptor for depth images, called histogram of depth differences (HDD). Second, we extend PS to 3D by proposing 3D pairwise constraints and a new method that makes exact inference tractable. Our approach is evaluated for pose estimation and clinician detection on a challenging RGB-D dataset recorded in a busy operating room during live surgeries. We conduct series of experiments to study the different part detectors in conjunction with the various 2D or 3D pairwise constraints. Our comparisons demonstrate that 3D PS with RGB-D part detectors significantly improves the results in a visually challenging operating environment.Comment: The supplementary video is available at https://youtu.be/iabbGSqRSg

Méthodes de radioprotection réactives au contexte pour la salle d’opération hybride

The use of X-ray imaging technologies during minimally-invasive procedures exposes both patients and medical staff to ionizing radiation. Even if the dose absorbed during a single procedure can be low, long-term exposure can lead to noxious effects (e.g. cancer). In this thesis, we therefore propose methods to improve the overall radiation safety in the hybrid operating room by acting in two complementary directions. First, we propose approaches to make clinicians more aware of exposure by providing in-situ visual feedback of the ongoing radiation dose by means of augmented reality. Second, we propose to act on the X-ray device positioning with an optimization approach for recommending an angulation reducing the dose deposited to both patient and staff, while maintaining the clinical quality of the outcome image. Both applications rely on approaches proposed to perceive the room using RGBD cameras and to simulate in real-time the propagation of radiation and the deposited dose.L’utilisation de systèmes d’imagerie à rayons X lors de chirurgies mini-invasives expose patients et staff médical à des radiations ionisantes. Même si les doses absorbées peuvent être faibles, l’exposition chronique peut causer des effets nocifs (e.g. cancer). Dans cette thèse, nous proposons des nouvelles méthodes pour améliorer la sécurité vis-à-vis des radiations ionisantes en salle opératoire hybride dans deux directions complémentaires. Premièrement, nous présentons des approches pour rendre les cliniciens plus conscients des irradiations grâce à des visualisations par réalité augmentée. Deuxièmement, nous proposons une méthode d'optimisation pour suggérer une pose de l’imageur réduisant la dose au personnel et patient, tout en conservant la qualité de l’image. Pour rendre ces applications possibles, des nouvelles approches pour la perception de la salle grâce à des caméras RGBD et pour la simulation en temps-réel de la propagation et doses de radiation ont aussi été proposées

Evaluation of occupational exposure to static magnetic field in MRI sites based on body pose estimation and SMF analytical computation

International audienceThis paper tackles the problem of estimating exposure to static magnetic field (SMF) in magnetic resonance imaging (MRI) sites using a non‐invasive approach. The proposed approach relies on a vision‐based system to detect people's body parts and on a mathematical model to compute SMF exposure. A multi‐view camera system was used to capture the MRI room, and a vision‐based system was applied to detect body parts. The detected localization was then fed into a mathematical model to compute SMF exposure. In this study, we focused on exposure at the neck due to two main reasons. First, according to regulations, the limit of exposure at head and trunk for MR workers is higher than that for the general public. Second, it was easier to attach a dosimeter at the neck to perform measurements, which allowed a quantitative evaluation of our approach. This approach was applied to two scenarios simulating the daily movements of medical workers for which dosimeter measurements were also recorded. The results indicated that the proposed approach predicted occupational SMF exposure with reasonable accuracy compared with the dosimeter measurements. The proposed approach is a simple safe working procedure to estimate the exposure of MR workers at different parts of the body without wearing any marker detection. It can be applied to reduce occupational SMF exposure, without changes in workers’ performances. For that reason, our non‐invasive proposed method can be used as a simple safety tool to estimate occupational SMF exposure in MR sites